To satisfy the ever-growing need for computing power, the computing industry has moved towards the use of distributed computing systems. In a distributed computing system, a plurality of computing nodes are coupled together via a network to form an overall more powerful system. One of the advantages of a distributed computing system is that it is highly scalable. To increase the computing power of the system, one or more computing nodes may simply be added. Another advantage of a distributed system is that it enables less expensive, commodity computing nodes to be used. This makes it possible to add computing power to a system with relatively minimal cost. Because of these and other advantages, the popularity of distributed computing systems has grown in recent years.
In a distributed computing system, one of the major considerations is the ability of the various computing nodes to communicate with each other. The more easily and efficiently the computing nodes can communicate and interact, the more the overall system appears to be a single integrated system. One of the aspects of this node-to-node interaction is the ability of one node to access the memory controlled by another node.
Currently, a local node can access the memory controlled by a remote node in several ways. One way is through the use of programmed I/O (input/output). With programmed I/O, a portion of the memory controlled by the remote node is mapped into the physical address space of the local node. Once mapped in this way, a processor of the local node may access the remote memory portion as if it were a part of the local node's local memory. The processor may do this, for example, by issuing memory transaction messages (MTMs). These MTMs may have the same format and conform to the same processor bus protocol as the MTMs that the processor would otherwise issue to access the local node's local memory. An underlying component (for example, a network interface) would encapsulate these MTMs within network packets, and send those packets across the network to the remote node. In turn, the remote node would process the packets and perform the requested accesses on the memory that it controls. In this way, the processor on the local node is able to access the memory controlled by the remote node.
With programmed I/O, the processor of the local node expects the same operation and result from a remote memory access as it does from a local memory access. Thus, when performing a remote memory access, the network needs to ensure that its behavior satisfies the expectations of the processor. If it does not, serious errors may result. With programmed I/O, the processor has two main expectations. First, the processor expects the MTMs that it issues will be processed in an order that is consistent with the processor bus protocol. This ordering is important in ensuring proper processing of information, deadlock avoidance, etc. Second, the processor expects that its MTMs will be processed. The MTMs cannot be dropped or ignored. In order to accommodate remote memory access, a network needs to guarantee that these two expectations are met.
Unfortunately, most standard commodity networks, such as Ethernet, do not satisfy these conditions. In an Ethernet network, for example, the switches within the network, under certain circumstances, may drop packets. In the context of programmed I/O, such dropped packets may, and most likely will, lead to serious errors. Also, because packets may be dropped in an Ethernet network, there is no guarantee that packets will be received and processed in any particular order (even if dropped packets are resent). As a result, it has thus far not been possible to use standard commodity networks to implement remote programmed I/O. Rather, proprietary networks such as SCI and DEC memory channel have been used. These proprietary networks are undesirable, however, because they tend to be expensive. Also, because they are non-standard, they tend to be incompatible with most standard equipment, which leads to more increased cost. Because of these and other shortcomings of proprietary networks, it has been difficult up to this point to implement remote programmed I/O in a cost effective and efficient manner.